Clean up old generations that are building up in active long-running workflows - #415

Summary

Prune old execution generations from long-running workflows, including workflows that are still active.

Previously, continue_as_new bounded replay history but not stored history: old generations remained until the entire workflow was deleted. Background reconciliation now removes completed, non-current generations older than pg_durable.retention_days.

Depends on microsoft/duroxide-pg#32.

Changes

• Integrate the provider’s cursor-based batch-pruning API into background maintenance, covering control, satellite, and nested-loop executions.
• Preserve current and Running executions, workflow metadata, and current results.
• Bound each slice to at most 1,000 scanned execution keys, including protected or ineligible rows, so scanning continues through retained history.
• Use maintenance connections with server-side statement and lock deadlines rather than consuming the runtime’s connection pool.
• Retry cancelled queries with smaller batches after rollback, allowing individually prunable generations to progress when their combined batch exceeds the deadline. Other failures and unprunable single generations are deferred without blocking later pages.
• Read execution-history details and event counts in one snapshot, avoiding lookup errors when pruning runs concurrently.
• Update worker readiness, test-runner integration, retention documentation, and upgrade guidance.

Retention and upgrade behavior

• Historical generations use the existing age-based retention window, not a generation-count cap.
• pg_durable.reconcile_interval = 0 also disables execution pruning.
• The worker applies provider migrations before publishing readiness. The provider recovers missing completion timestamps from recorded terminal events; missing events retain NULL timestamps with a warning, while malformed history aborts migration with diagnostic keys.
• Upgrade requires draining all old provider callers, including external clients and maintenance jobs—not only workers. Backfill can increase startup time and WAL volume.
• No extension SQL upgrade is required for this feature; supported older df schemas remain compatible.
• Pruning does not cap history within a single execution or reclaim completed child instances independently of their owning workflow.

Landing blocker: the provider remains pinned to immutable revision 7987089 with DONOTLAND. Replace it with an exact compatible released version once the upstream changes are published. Earlier pre-release builds used the same migration numbers with different bodies; updating their dependency alone does not reinstall those functions.